Image decoding apparatus

ABSTRACT

An image decoding apparatus is capable of decoding coded bit streams with different coding schemes. The image decoding apparatus includes a coding scheme decision section for deciding a coding scheme from coding scheme identification information multiplexed into a coded bit stream, a setting unit for setting header information on a second coding scheme in accordance with header information in a first coding scheme, and a decoder for decoding image coded data in the first coding scheme in response to the header information on the second coding scheme, which is set.

This application is a Divisional of co-pending application Ser. No.11/016,889 filed on Dec. 21, 2004 and for which priority is claimedunder 35 U.S.C. §120. Application Ser. No. 11/016,889 is also adivisional application of U.S. patent application Ser. No. 09/529,304filed on Apr. 12, 2000, which is the national phase under 35 U.S.C. §371of PCT International Application No. PCT/JP97/03846 which has anInternational filing date of Oct. 23, 1997, which designated the UnitedStates of America. The entire contents of each of the above-identifiedapplications are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to an image decoding apparatus, an imagecoding apparatus, an image communications system and a coded bit streamconverting apparatus, all of which can handle coded bit streams withdifferent coding schemes.

BACKGROUND ART

A system based on the MPEG-4 (Moving Picture Experts Group Phase-4)which is currently in progress toward standardization in ISO/IECJTC11/SC29/WG11 differs from a system based on ITU-T RecommendationH.263 in header information (an information signal for decoding) to beadded to a coded bit stream which constitutes a transmitted signal.

FIG. 1( a) is a diagram showing a structure of an H.263 coded bit stream201 based on the H.263 standard, into which header information 211 ismultiplexed along with macroblock data 225 consisting of image codeddata encoded according to the H.263 coding scheme. FIG. 1( b) is adiagram showing a structure of an MPEG-4 coded bit stream 202, intowhich header information 212 is multiplexed along with macroblock data239 consisting of image coded data encoded according to the MPEG-4coding scheme. As shown in these figures, they have different coded bitstream structures. In particular, the H.263 does not include headerinformation about VO (Video Object), VOL (Video Object Layer), VOP(Video Object Plane) and the like, which are required for MPEG-4decoding. Accordingly, to carry out the image communications based onthe two schemes, separate image decoding apparatuses and image codingapparatuses are needed.

Here, it is not always necessary for a GOB start code 223 and GOB headerinformation 224 in the H.263 coded bit stream 201, and resynchronizationinstruction code 237 and resynchronization information 238 in the MPEG-4coded bit stream 202 to be inserted, but inserted as needed.

With such structures, the conventional coded bit streams present aproblem in that an MPEG-4 compatible image decoding apparatus, forexample, cannot decode the H.263 coded bit stream 201 generatedaccording to the H.263 standard.

Furthermore, to decode the coded bit streams according to the MPEG-4 andH.263 standard, an image decoding apparatus must comprise two decodersbased on the two schemes, which presents a problem of complicating theapparatus.

JP-9139937 already discloses a moving picture stream convertingapparatus which is constructed by demultiplexer, a packetizer, and amultiplexer. The demultiplexer receives an MPEG 1 system streamincluding a stream of coded video data and a stream of encoded stream ofencoded video data and a stream of encoded audio data and separates anelementary system (ES) of each of the video and audio data. A packetizedelementary stream PES of MPEG 2 is formed by the packetizer from each ofthe video and audio elementary streams separated by the demultiplexer.The multiplexer divides the video and audio packetized elementarystreams into transport packets each having a fixed length (188 bytes)and, thereafter, multiplexes the transport packets, thereby convertingto a transport stream (hereinafter, referred to as “TS”) of MPEG 2. TheMPEG 1 system stream as a conversion source is constructed by aplurality of packs, in which one pack obtained by combining a pluralityof packs, in which one page obtained by combining a plurality of packetsis set to one unit. A head pack is constructed by a pack header, asystem header, and a plurality of packets. Each of second and subsequentpacks is constructed by an ES header and an ES payload. A packed startcode, a stream ID indicative of the king of video/audio and a channelnumber, a packet length of ES payload, time management information of areproduction and time management information of a decoding have beenstored as parameters in the elementary stream header. Video or audioencoded data has been stored in the ES payload. Each packet of the PESof MPEG 2 is constructed by a PES header and a PES payload. A packetstart code, a stream ID indicative of the king of video/audio and achannel number, a packet length of PES payload, time managementinformation of a reproduction output, and time management information ofa decoding have been stored in the PES header. Video or audio encodeddata has been stored in the PES payload.

The present invention is implemented to solve the foregoing problems.Therefore, an object of the present invention is to provide an imagedecoding apparatus capable of decoding the H.263 coded bit stream 201,an image coding apparatus for generating a coded bit stream decodable bythe image decoding apparatus, and an image communications system and acoded bit stream converting apparatus for converting the H.263 coded bitstream to the MPEG-4 coded bit stream to carry out communication, all ofwhich apparatuses have a simple structure.

DISCLOSURE OF THE INVENTION

According to the present invention, there is provided an image decodingapparatus for decoding at least a first coded bit stream into whichheader information of the H.263 coding scheme and image coded dataencoded in a the H.263 coding scheme are multiplexed, or for decoding asecond coded bit stream into which header information of the MPEG-4coding scheme and image coded data encoded in the MPEG-4 coding schemeare multiplexed, the image decoding apparatus comprising: coding schemedecision means for making a decision as to whether a received coded bitstream is the first coded bit stream or the second coded bit stream inresponse to the first header information or to the second headerinformation; decoding means for decoding image coding informationincluded in the header information of the MPEG-4 coding scheme byreceiving the second coded bit stream; and setting means for setting VOLgeometry information in image coding information to be used for decodingthe received first coded bit stream according to a syntax of the secondcoded bit stream to a rectangle shape, wherein the image decodingapparatus decodes the image coded data included in the first coded bitstream or in the second coded bit stream in response to the image codinginformation set by the setting means or in response to the image codinginformation decoded by the decoding means.

In the image decoding apparatus in accordance with the presentinvention, the coding scheme decision means can make the decision inresponse to coding scheme identification information for identifying thefirst or second coding scheme, the coding scheme identificationinformation being included in the first header information or in thesecond header information.

This offers an advantage of being able to decode the coded bit streamsbased on the different coding schemes with readily identifying thecoding schemes.

In the image decoding apparatus in accordance with the presentinvention, the coding scheme decision means can make the decision inresponse to a start code included in the first header information or inthe second header information.

This offers an advantage of being able to decode the coded bit streamsbased on the different coding schemes with readily identifying thecoding schemes.

In the image decoding apparatus in accordance with the presentinvention, the coding scheme decision means can make the decision inresponse to an H.263 start code included in the first headerinformation, or to a VOL (Video Object Layer) start code included in thesecond header information.

This offers an advantage of being able to decode the coded bit streamsbased on the H.263 and MPEG-4 coding schemes with readily identifyingthe coding schemes.

In the image decoding apparatus in accordance with the presentinvention, the coding scheme decision means can make the decision inresponse to a picture start code included in the first headerinformation, or to a VO (Video Object) start code included in the secondheader information.

This offers an advantage of being able to decode the coded bit streamsbased on the H.263 and MPEG-4 coding schemes with readily identifyingthe coding schemes without adding new header information.

According to a second aspect of the present invention, there is providedan image coding apparatus comprising: coding means for generating afirst coded bit stream by encoding an image signal in a first codingscheme; and header information multiplexing means for multiplexing, intothe first coded bit stream, header information for ensuringcompatibility with a second coded bit stream encoded in a second codingscheme.

This offers an advantage of being able to generate the first coded bitstream that can be decoded by the decoder for decoding the second codedbit stream.

In the image coding apparatus according to the present invention, theheader information multiplexing means can multiplex, as the headerinformation for ensuring the compatibility, a start code of the secondcoding scheme, and coding scheme identification information indicativeof the first coding scheme.

This offers an advantage of being able to generate the first coded bitstream the decoder for decoding the second coded bit stream can decodewith readily identifying the coding scheme.

According to a third aspect of the present invention, there is providedan image communications system comprising: coding means for generating afirst coded bit stream by encoding an image signal in a first codingscheme; decoding means for decoding a second coded bit stream coded in asecond coding scheme; and coded bit stream converting means fortransmitting the first coded bit stream received from the coding meansto the decoding means, after multiplexing into the first coded bitstream header information for ensuring compatibility, which is receivedfrom the decoding means.

This offers an advantage of being able to generate the first coded bitstream the decoder for decoding the second coded bit stream can decodewith readily identifying the coding scheme.

According to a fourth aspect of the present invention, there is provideda coded bit stream converting apparatus comprising: syntax analyzingmeans for inputting a first coded bit stream generated in a first codingscheme, and for extracting first header information in the first codingscheme and image coded data; decoding means for decoding the firstheader information extracted; header information setting means forsetting and coding second header information in a second coding schemein response to the first header information decoded by the decodingmeans; and multiplexing means for generating a second coded bit streamby multiplexing image coded data extracted by the syntax analyzing meanswith the second header information coded by the header informationsetting means.

This offers an advantage of being able to readily convert the firstcoded bit stream to the second coded bit stream.

According to a fifth aspect of the present invention, there is providedan image decoding apparatus for decoding a first coded bit stream intowhich first header information and image coded data encoded in a firstcoding scheme are multiplexed, or for decoding a second coded bit streaminto which second header information and image coded data encoded in asecond coding scheme are multiplexed, the image decoding apparatuscomprising: coding scheme decision means for making a decision as towhether a received coded bit stream is the first coded bit stream or thesecond coded bit stream in response to the first header information orto the second header information; first decoding means for decoding thefirst header information by receiving the first coded bit stream; andsecond decoding means for decoding image coding information on thesecond coding scheme included in the second header information byreceiving the second coded bit stream, wherein the image decodingapparatus decodes, when the coded bit stream received is the first codedbit stream, the image coded data included in the first coded bit streamin response to the first header information decoded by the firstdecoding means, and decodes, when the coded bit stream received is thesecond coded bit stream, the image coded data included in the secondcoded bit stream in response to the image coding information decoded bythe second decoding means.

This offers an advantage of being able to decode the coded bit streamsbased on the different coding schemes with readily identifying thecoding schemes without adding new header information, and to decode thefirst coded bit stream without setting the image coding information onthe second coding scheme.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a structure of a conventional H.263 codedbit stream and a structure of an MPEG-4 coded bit stream;

FIG. 2 is a diagram showing structures of a coded bit stream received bythe image decoding apparatus of an embodiment 1 in accordance with thepresent invention;

FIG. 3 is a block diagram showing a configuration of the image decodingapparatus of the embodiment 1 in accordance with the present invention;

FIG. 4 is a block diagram showing a configuration of the syntaxanalysis/variable length decoder in the embodiment 1 in accordance withthe present invention;

FIG. 5 is a block diagram showing a configuration of the headerinformation analyzer in the embodiment 1 in accordance with the presentinvention;

FIG. 6 is a block diagram showing a configuration of the H.263 pictureheader information analyzer in the embodiment 1 in accordance with thepresent invention;

FIG. 7 is a block diagram showing a configuration of the H.263 pictureheader information decoder in the embodiment 1 in accordance with thepresent invention;

FIG. 8 is a block diagram showing a configuration of the H.263 GOBheader information analyzer in the embodiment 1 in accordance with thepresent invention;

FIG. 9 is a diagram illustrating a GOB;

FIG. 10 is a block diagram showing a configuration of the GOB headerinformation decoder in the embodiment 1 in accordance with the presentinvention;

FIG. 11 is a diagram showing a layer structure of H.263 macroblock data;

FIG. 12 is a block diagram showing a configuration of the macroblocklayer syntax analyzer in the embodiment 1 in accordance with the presentinvention;

FIG. 13 is a block diagram showing a configuration of the block datadecoder in the embodiment 1 in accordance with the present invention;

FIG. 14 is a diagram illustrating calculation of a prediction vector;

FIG. 15 is a block diagram showing a configuration of the texturedecoder in the embodiment 1 in accordance with the present invention;

FIG. 16 is a block diagram showing a configuration of the inversequantizer in the embodiment 1 in accordance with the present invention;

FIG. 17 is a block diagram showing a configuration of an image codingapparatus of embodiments 2 and 4 in accordance with the presentinvention;

FIG. 18 is a diagram illustrating a relationship between an H.263encoder and an MPEG-4 decoder in the embodiments 2 and 4 in accordancewith the present invention;

FIG. 19 is a diagram showing contents of an MPEG-4 compatible H.263coded bit stream in an embodiment 3 in accordance with the presentinvention;

FIG. 20 is a block diagram showing a configuration of a headerinformation analyzer in the embodiment 3 in accordance with the presentinvention;

FIG. 21 is a diagram showing an image communications system of anembodiment 5 in accordance with the present invention;

FIG. 22 is a diagram showing an image communications system of anembodiment 6 in accordance with the present invention;

FIG. 23 is a block diagram showing a configuration of a headerinformation analyzer of an embodiment 7 in accordance with the presentinvention 7;

FIG. 24 is a diagram illustrating the start and end of a coded bitstream in the embodiment 7 in accordance with the present invention;

FIG. 25 is a block diagram showing a coded bit stream convertingapparatus of an embodiment 8 in accordance with the present invention;and

FIG. 26 is a diagram showing a structure of GOB header information and astructure of resynchronization information.

BEST MODE FOR CARRYING OUT THE INVENTION

The invention will now be described in more detail with reference to theaccompanying drawings.

Embodiment 1

FIG. 2 is a diagram showing structures of a coded bit stream received byan image decoding apparatus of an embodiment 1, wherein FIG. 2( a) showsan MPEG-4 compatible H.263 coded bit stream 203, and FIG. 2( b) shows anMPEG-4 coded bit stream 204. The MPEG-4 compatible H.263 coded bitstream 203 as shown in FIG. 2( a) includes, in addition to theconventional H.263 coded bit stream 201 as shown in FIG. 1( a), a VOstart code 231, a VO identification number 232, a VOL start code 233 andH.263 compatible identification information 226. The MPEG-4 coded bitstream 204 as shown in FIG. 2( b) includes, in addition to theconventional MPEG-4 coded bit stream 202 as shown in FIG. 1( b), H.263compatible identification information 226. The H.263 compatibleidentification information 226 added to the MPEG-4 compatible H.263coded bit stream 203 is distinguishable from that added to the MPEG-4coded bit stream 204 because one of the H.263 compatible identificationinformation is placed at “0”, and the other information at “1”.

FIG. 3 is a block diagram showing a configuration of an image decodingapparatus for decoding a VO (Video Object) in the embodiment 1. In FIG.3, the reference numeral 1 designates a received coded bit stream; and 2designates a syntax analysis/variable length decoder that analyzes inthe coded bit stream 1 syntax (a multiplexed video signal), and outputsgeometry coded data 3, texture coded data 6 and texture motion data 7.The reference numeral 4 designates a geometry decoder for obtainingdecoded geometry data 5 by decoding the geometry coded data 3; 8designates a motion compensator for carrying out motion compensation inresponse to the texture motion data 7 to obtain prediction texture data9; and 10 designates a texture decoder for carrying out decoding inresponse to the texture coded data 6 and prediction texture data 9 toobtain decoded texture data 11.

Next, the operation will be described.

Here, decoding operation of the MPEG-4 compatible H.263 coded bit stream203 as shown in FIG. 2( a), which is a subject matter of the presentinvention, will be chiefly described. In other words, a case will bedescribed, in which shapes of individual VOPs are rectangular, that is,no bit stream includes geometry coded data, and the texture data orinformation about motion is encoded on a macroblock basis.

Incidentally, the basic operation for decoding the MPEG-4 coded bitstream 204 as shown in FIG. 2( b) is the same as the conventionaloperation.

First, the syntax analysis/variable length decoder 2 translates theinput coded bit stream 1 from a binary bit stream to intelligible data.Thus, the syntax analysis/variable length decoder 2 enables the MPEG-4compatible H.263 coded bit stream 203 to be decoded. The motioncompensator 8 carries out the motion compensation in response to thetexture motion data 7 output from the syntax analysis/variable lengthdecoder 2, and outputs the prediction texture data 9. The texturedecoder 10 receives the texture coded data 6 output from the syntaxanalysis/variable length decoder 2 and the prediction texture data 9output from the motion compensator 8, and obtains the decoded texturedata 11.

Next, the operation of the syntax analysis/variable length decoder 2will be described.

FIG. 4 is a block diagram showing a configuration of the syntaxanalysis/variable length decoder 2. In this figure, the referencenumeral 21 designates a header information analyzer for extracting theheader information added to the coded bit stream 1, and for settingvarious pieces of header information required for the subsequentdecoding control; 22 designates a macroblock layer syntax analyzer forobtaining the texture coded data 6 and texture motion data 7 from thecoded bit stream 1.

FIG. 5 is a block diagram showing a configuration of the headerinformation analyzer 21. In this figure, the reference numeral 30designates a VO start code detector coded for detecting the VO startcode 231 in the bit stream 1; 31 designates a VOL start code detectorfor detecting the VOL start code 233 from the coded bit stream 1; and 32designates a coding scheme decision section for making a decision as towhether the coded bit stream 1 is the MPEG-4 compatible H.263 coded bitstream 203 or the MPEG-4 coded bit stream 204, and for outputting H.263compatible identification information 33. The reference numeral 34designates a switching section switched in response to the decidedcoding scheme; 35 designates an H.263 picture header informationanalyzer for decoding from the MPEG-4 compatible H.263 coded bit stream203 the picture header information 222 which is the image codinginformation unique to the H.263 system, and for setting the VOL headerinformation 234 and VOP header information 236 which are the imagecoding information unique to the MPEG-4 system; 36 designates an H.263GOB header information analyzer for decoding from the MPEG-4 compatibleH.263 coded bit stream 203 the H.263 GOB (Group of Block) headerinformation 224, and for updating, in response to the GOB headerinformation 224 decoded, the VOP header information 236 set by the H.263picture header analyzer 35; 37 designates a VOL header informationdecoder for decoding the VOL header information 234 from the MPEG-4coded bit stream 204; and 38 designates a VOP header informationanalyzer for decoding the VOP header information 236 from the MPEG-4coded bit stream 204.

Next, the operation of the header information analyzer 21 will bedescribed.

Detecting the VO start code 231 in the MPEG-4 compatible H.263 coded bitstream 203 or in the MPEG-4 coded bit stream 204 as shown in FIG. 2, theVO start code detector 30 starts the following decoding operation.Specifically, the VOL start code detector 31 detects the VOL start code233 in the coded bit stream 1. The coding scheme decision section 32decodes from the coded bit stream 1 the H.263 compatible identificationinformation 226, and makes a decision from the H.263 compatibleidentification information 226 as to whether the coded bit stream 1 isthe MPEG-4 compatible H.263 coded bit stream 203 or the MPEG-4 coded bitstream 204, thereby outputting the H.263 compatible identificationinformation 33.

When the coded bit stream 1 is the MPEG-4 compatible H.263 coded bitstream 203, the switching section 34 supplies the coded bit stream 1 tothe H.263 picture header information analyzer 35.

FIG. 6 is a block diagram showing a configuration of the H.263 pictureheader information analyzer 35. When an H.263 picture start codedetector 41 detects the picture start code 221 in the coded bit stream1, a subsequent H.263 picture header information decoder 42 decodes thepicture header information 222 from the coded bit stream 1. Then, anMPEG-4 header information setting section 43 sets the VOL headerinformation 234 and VOP header information 236 in response to thepicture header information 222 decoded.

FIG. 7 is a block diagram showing a configuration of the H.263 pictureheader information decoder 42. A temporal reference (TR) decoder 51receives the bit stream 1 from the H.263 picture start code detector 41,and decodes the number of pictures (TR) that are skipped or not referredto among the transmitted pictures. This information is used for displayas needed.

Next, a picture type (PTYPE) decoder 52 decodes the picture type(PTYPE). The picture type includes information such as a picture format301, a picture coding type 302 and an optional mode indication flag 303.The picture format 301 and picture coding type 302 decoded are suppliedto the MPEG-4 header information setting section 43 shown in FIG. 6.

The picture type (PTYPE) decoder 52 makes a decision as to whether theoptional mode indication flag 303 is ON or not. Although the H.263standard offers several optional modes, the image decoding apparatusdescribed in the present embodiment does not ensure the compatibilitybetween bit streams containing the optional modes. Thus, the coded bitstream with the optional mode being ON (valid) is supplied to a decodingoperation terminating section 54 through a switching section 53 so thatthe decoding operation terminating section 54 completes the decodingoperation of the coded bit stream. The picture type includes informationdefining display or others, which are available as need.

In contrast, the bit stream with the optional mode being OFF (invalid)is supplied to a picture quantization step size (PQUANT) decoder 55through the switching section 53. The picture quantization step size(PQUANT) decoder 55 decodes a picture quantization step size (PQUANT)304. The picture quantization step size 304 decoded is supplied to theMPEG-4 header information setting section 43 shown in FIG. 6. Thepicture header information after the picture quantization step size 304is skipped because it is not required in the subsequent decoding.

Next, the operation of the MPEG-4 header information setting section 43as shown in FIG. 6 will be described.

The MPEG-4 header information setting section 43 sets, in response tothe picture header information 222 decoded, VOL geometry information andobject size as the VOL header information 234. It also sets, in the caseof the MPEG-4 compatible H.263 coded bit stream, the informationindicating that the geometry information represents rectangles, in whichcase, the individual bit streams correspond to frames each. Furthermore,since the object size corresponds to the frame size, the MPEG-4 headerinformation setting section 43 obtains the frame size from the pictureformat 301, one of the picture header information 222, and sets theobject size. In addition, it also sets information about whether thegray scale per pixel is eight bits or not. Because the H.263 systemassumes that the gray scale per pixel is always eight bits, it is placedat eight bits.

Next, the MPEG-4 header information setting section 43 invalidates theMPEG-4 based coding conditions, that is, the sprite coding, errorresistant coding, intra AC/DC prediction and scalability coding. Becausethe MPEG-4 can select its quantization scheme from the two schemes H.263and MPEG-1/2, the quantization scheme is set in advance at the H.263when using the MPEG-4 compatible H.263 coded bit stream 203.

Furthermore, the MPEG-4 header information setting section 43 sets theVOP header information 236. Specifically, it sets as the VOP headerinformation 236, the VOP prediction type information and quantizationstep size. The VOP prediction type includes intra coding that uses onlythe data within the VOP, and inter coding that also uses data before andafter the VOP. The VOP prediction type information is set in response toa picture coding type 302, one of the picture header information 222.Besides, the VOP quantization step size is set in accordance with apicture quantization step size 304, one of the picture headerinformation 222.

Moreover, because the MPEG-4 can select its motion vector search rangefrom seven types, it has a code for designating the motion vector searchrange. However, since the H.263 corresponds to only one of the searchranges, it is necessary for the MPEG-4 header information settingsection 43 to set the motion vector search range designation codecorresponding to the motion vector search range the H.263 employs. Inaddition, although the MPEG-4 is interlace image compatible, the H.263is interlace incompatible. Thus, interlace mode indication informationis always set invalid.

After the H.263 picture header information analyzer 35 as shown in FIG.5 completes the analysis of the picture header information, the H.263GOB header information analyzer 36 starts the analysis of the GOB headerinformation 224, when the coded bit stream includes the GOB start code223 and GOB header information 224. When the coded bit stream does notinclude the GOB start code 223 or GOB header information 224, the H.263GOB header information analyzer 36 does not operate.

FIG. 8 is a block diagram showing a configuration of the H.263 GOBheader information analyzer 36 as shown in FIG. 5. When a GOB start codedetector 61 detects the GOB start code 223 attached to the coded bitstream 1, a GOB header information decoder 62 decodes the GOB headerinformation 224.

FIG. 9 illustrates GOBs. As shown in this figure, each GOB includes aseries of macroblocks formed by dividing an image, and the GOB headerinformation 224 includes information required for establishingresynchronization at a decoding side. A bit error in the coded bitstream will propagate to subsequent macroblock data in the case ofvariable length coding or prediction coding, thereby impairing correctdecoding. The detection of the GOB header information can prevent thepropagation of the error because it establishes the resynchronization ofthe coded bit stream before decoding the initial macroblock of the GOB,and thus resets the information needed for decoding the successivemacroblocks. The quantization step size and the motion vector of eachmacroblock must be reset when the resynchronization is established,because they undergo the prediction coding that codes the differencesbetween the quantization step sizes and between the motion vectors ofthe current and previous coded macroblocks.

FIG. 10 is a block diagram showing a configuration of a GOB headerinformation decoder 62. A GOB number decoder 71 decodes a GOB number(GN) from the coded bit stream 1. A GOB frame identification numberdecoder 72 decodes the identification number (GFID) of a picture towhich the GOB belongs. A GOB quantization step size decoder 73 decodes aGOB quantization step size (GQUANT) 305, and supplies it to an MPEG-4header information update section 63 as shown in FIG. 8.

The MPEG-4 header information update section 63 updates, in response tothe decoded GOB header information 224, the VOP header information 236set by the MPEG-4 header information setting section 43. It is thequantization step size that is updated in response to the GOB headerinformation 224. Thus, the GOB quantization step size is placed at theVOP quantization step size. The foregoing pieces of information that areset are supplied to the macroblock layer syntax analyzer 22 as shown inFIG. 4.

The coding scheme decision section 32 as shown in FIG. 5 makes adecision, when the H.263 compatible identification information 226indicates the MPEG-4, that the coded bit stream 1 is the MPEG-4 codedbit stream 204, and outputs the H.263 compatible identificationinformation 33. The MPEG-4 coded bit stream 204 is supplied to the VOLheader information decoder 37 through the switching section 34. The VOLheader information decoder 37 decodes the VOL header information 234from the coded bit stream, and the VOP header information analyzer 38decodes the VOP header information 236, and supplies it to themacroblock layer syntax analyzer 22 of FIG. 4.

After setting the foregoing information, the macroblock layer syntaxanalyzer 22 decodes the macroblock data 225 or 239 through the analysisbased on the MPEG-4 syntax. However, since the coding scheme of theblock data differs a little between the MPEG-4 and H.263, the decodingside must also switch the operation mode.

FIG. 11 is a diagram showing a layer structure of the macroblock data225 in the MPEG-4 compatible H.263 bit stream 203 in the presentembodiment 1. The macroblock consists of four luminance blocks and twocolor difference blocks. As shown in FIG. 11, each macroblock includesmacroblock skip decision information 251, macroblock type/valid colordifference block identification information 252, valid blockidentification information 253, differential quantization step size 254and motion data 255, which are multiplexed as attribute information.

Here, the macroblock skip decision information 251 indicates whether themotion vector is zero and all the coefficient data within the macroblockin the inter VOP are zero (the coefficient data are obtained by passingthe input image signal (the original signal when intra coded, and thedifferential signal between it and a reference VOP when inter coded)through the DCT, and then through the quantization). When the motionvector is zero and all the coefficient data are zero, the subsequentinformation about the macroblock is excluded from the bit stream, andskipping to the next macroblock is carried out.

The macroblock type in the macroblock type/valid color difference blockidentification information 252 indicates a macroblock coding type whenthe macroblock data is coded using the original signal of the macroblock(intra), or when the differential signal between the macroblock and thereference macroblock is coded after the motion compensation prediction(inter), or when the current macroblock is coded using the quantizationstep size different from the quantization step size of the immediatelyprevious macroblock.

The valid block identification information 253 indicates whether thecoefficient data of the blocks are all zero or not. Although theforegoing attribute information is followed by coefficient data(corresponding to block data 256) multiplexed into each block, thecoefficient data of the block is absent when the valid blockidentification information 253 indicates that it is a invalid block.

The differential quantization step size 254 is information multiplexedwhen the macroblock type indicates that the quantization step size ofthe current macroblock differs from that of the immediately precedingmacroblock, and indicates the differential value from the quantizationstep size of the preceding macroblock.

FIG. 12 is a block diagram showing a configuration of the macroblocklayer syntax analyzer 22. In this figure, the reference numeral 81designates a switching section that is switched in response to geometryinformation 311 set by the MPEG-4 header information setting section 43;82 designates a geometry coded data decoder for decoding the geometrycoded data in the coded bit stream; 83 designates a switching sectionthat is switched in response to VOP prediction type 312 set by theMPEG-4 header information setting section 43; 84 designates a skipdecision information decoder for decoding, when the VOP prediction typeis other than the intra coding, the macroblock skip decision information251; 85 designates a switching section that is switched in response tothe skip decision information 251; 86 designates a skip associated datasetting section for placing all the motion vector and texture data inthe macroblock to zero when skipping; and 87 designates a macroblocktype/valid color difference block identification information decoder fordecoding macroblock type 313 and valid color difference blockidentification information when the VOP prediction type 312 is intra orthe skipping is not carried out.

The reference numeral 88 designates a switching section that is switchedin response to intra AC/DC prediction mode indication information 315set by the MPEG-4 header information setting section 43; 89 designatesan AC prediction indication information decoder for decoding ACprediction indication information; 90 designates a valid blockidentification information decoder for decoding the valid blockidentification information 253; and 91 designates a switching sectionthat is switched in response to the macroblock type 313 output from themacroblock type/valid color difference block identification informationdecoder 87.

The reference numeral 92 designates a differential quantization stepsize zero setting section for placing the differential quantization stepsize to zero; 93 designates a differential quantization step sizedecoder for decoding a differential quantization step size 317; 94designates an adder for adding the differential quantization step size317 and a VOP quantization step size 318 of the previous block, andsupplies a quantization step size 319 to the texture decoder 10 of FIG.3; 95 designates a switching section that is switched in response tointerlace mode indication information 316 fed from the MPEG-4 headerinformation setting section 43; 96 designates an interlace informationdecoder for decoding interlace information; 97 designates a motionvector decoder for decoding a motion vector (texture motion data 7) inresponse to the macroblock type 313 output from the macroblocktype/valid color difference block identification information decoder 87,to the VOP prediction type 312 output from the MPEG-4 header informationsetting section 43 and to motion vector search range designationinformation 320; and 98 designates a block data decoder for decoding thecoded block data, and supplies the texture coded data 6 to the texturedecoder 10.

Next, the operation of the macroblock layer syntax analyzer 22 will bedescribed.

The following description will be made for the coded bit stream 1consisting of the MPEG-4 compatible H.263 coded bit stream 203. As forthe MPEG-4 coded bit stream 204, the description will be omitted herebecause it is described in the ISO/IEC JTC1/SC29/WG11 MPEG-4 VideoVM8.0.

First, the switching section 81 switches the output of the coded bitstream 1 in response to the geometry information 311 set by the MPEG-4header information setting section 43. When the coded bit stream 1consists of the MPEG-4 compatible H.263 coded bit stream 203, thegeometry information 311 is rectangular, and hence the bit stream 1 isdirectly supplied to the switching section 83 without passing throughthe geometry coded data decoder 82.

Subsequently, the switching section 83 carries out its switching inresponse to the VOP prediction type 312 set by the MPEG-4 headerinformation setting section 43. When the VOP prediction type 312 isintra, the macroblock type/valid color difference block identificationinformation decoder 87 decodes the macroblock type 313 and the validcolor difference block identification information. When the VOPprediction type is other than intra, the skip decision informationdecoder 84 decodes the skip decision information 251 of the macroblock.The skip decision information 251 decoded switches the switching section85 such that when it indicates that the macroblock is to be skipped, theskip associated data setting section 86 places both the motion vector ofthe macroblock and the texture data in the macroblock all at zero, andcompletes the decoding of the macroblock. In contrast, when the skipdecision information 251 indicates that the macroblock must not beskipped, the macroblock type/valid color difference block identificationinformation decoder 87 decodes the macroblock type 313 and the validcolor difference block identification information.

Next, the switching section 88 is switched in response to the intraAC/DC prediction mode indication information 315 set by the MPEG-4header information setting section 43. As for the MPEG-4 compatibleH.263 coded bit stream 203, because it does not have a function to carryout the intra AC/DC prediction, and hence the intra AC/DC prediction isset invalid when setting the VOL header information 234, it is suppliedto the valid block identification information decoder 90 without passingthrough the AC prediction indication information decoder 89.

The valid block identification information decoder 90 decodes the validblock identification information 253 for the luminance block in themacroblock. The switching section 91 is switched in response to themacroblock type 313 decoded by the macroblock type/valid colordifference block identification information decoder 87 so that when thequantization step size of the instant macroblock differs from that ofthe first previous macroblock, the differential quantization step sizedecoder 93 decodes the differential quantization step size 317 betweenthe quantization step size of the instant macroblock and that of thefirst previous macroblock. The differential quantization step size 317decoded is added to the VOP quantization step size 318 of the firstprevious macroblock by the adder 94, and the sum is supplied to thetexture decoder 10 of FIG. 3 as the quantization step size 319.

In contrast, when the quantization step size of the current macroblockequals that of the first previous macroblock, the differentialquantization step size zero setting section 92 places the differentialquantization step size at zero.

Subsequently, the switching section 95 is switched in response to theinterlace mode indication information 316 fed from the MPEG-4 headerinformation setting section 43. As for the MPEG-4 compatible H.263 codedbit stream 203, because it does not correspond to the interlace image,the interlace mode is set invalid, and hence it is supplied to themotion vector decoder 97 without passing through the interlaceinformation decoder 96. The motion vector decoder 97 decodes, when theVOP prediction type 312 set by the MPEG-4 header information settingsection 43 is inter, the motion vector (texture motion data 7) inresponse to the macroblock type 313 decoded by the macroblock type/validcolor difference block identification information decoder 87 and to themotion vector search range designation information 320 set by the MPEG-4header information setting section 43, and supplies the motion vector tothe motion compensator 8 of FIG. 3.

Subsequently, the block data decoder 98 decodes the coded block data inthe coded bit stream. FIG. 13 is a block diagram showing a configurationof the block data decoder 98. In this figure, the reference numeral 101designates a switching section that receives the coded block data, andis switched in response to the macroblock type 313 fed from themacroblock type/valid color difference block identification informationdecoder 87; 102 designates a switching section that is switched inresponse to the intra AC/DC prediction mode indication information 315set by the MPEG-4 header information setting section 43; 103 designatesa DC coefficient fixed length decoder that carries out, when the intraAC/DC prediction is OFF, the DC coefficient fixed length decoding inresponse to gray scale per pixel 321 fed from the MPEG-4 headerinformation setting section 43, and outputs a decoded intra DCcoefficient 111; and 104 designates a DC coefficient decoder thatdecodes the DC coefficient when the intra AC/DC prediction is ON, andoutputs the decoded intra DC coefficient 111.

The reference numeral 105 designates a switching section that isswitched in response to the valid block identification information 253fed from the valid block identification information decoder 90; and 106designates an AC coefficient VLD table switching section for switchingan AC coefficient VLD (Variable Length Decoding) table in response tothe macroblock type 313 fed from the macroblock type/valid colordifference block identification information decoder 87 and to the H.263compatible identification information 33 fed from the coding schemedecision section 32.

The reference numeral 107 designates an AC coefficient data variablelength decoder that carries out the variable length decoding of the ACcoefficient data, and outputs decoded AC coefficient data 112; 108designates a switching section that is switched in response to the H.263compatible identification information 33 fed from the coding schemedecision section 32; 109 designates an AC coefficient data fixed lengthdecoder for outputting the decoded AC coefficient data 112; 110designates an AC coefficient data Esc coding decoder for outputting thedecoded AC coefficient data 112; and 113 designates an AC coefficientzero setting section for placing the AC coefficient at zero.

Next, the operation of the block data decoder 98 will be described.

First, the coded block data is switched by the switching section 101 inresponse to the macroblock type 313 fed from the macroblock type/validcolor difference block identification information decoder 87 such thatit is supplied to the switching section 105 when the macroblock type 313is other than intra. When the macroblock type 313 is intra, the codedblock data is supplied to the switching section 102 which is switched inresponse to the intra AC/DC prediction mode indication information 315set by the MPEG-4 header information setting section 43.

As for the MPEG-4 compatible H.263 coded bit stream 203, because theintra AC/DC prediction mode 315 is set invalid, it does not pass throughthe DC coefficient decoder 104, but is supplied to the DC coefficientfixed length decoder 103. The DC coefficient fixed length decoder 103carries out the fixed length decoding, and supplies the decoded intra DCcoefficient 111 to the texture decoder 10, and the coded block data tothe switching section 105. In this case, the length of the code passingthrough the fixed length decoding equals the gray scale per pixel (thedefault is 8 bits) 321 set by the MPEG-4 header information settingsection 43. Since the gray scale per pixel 321 has the default of 8bits, it equals that of the H.263 decoder.

The switching section 105 is switched in response to the valid blockidentification information 253 decoded by the valid block identificationinformation decoder 90 such that when the block is invalid, the ACcoefficient zero setting section 113 places the decoded AC coefficientdata 112 in the block at zero, and supplies it to the texture decoder10. When the block is valid, the coded block data is supplied to the ACcoefficient VLD table switching section 106.

The AC coefficients in the block undergo the variable length coding bythe encoder side that scans the coefficients in the block in apredetermined sequence, and encodes them with generating a combinationconsisting of a flag (LAST) indicating whether a non-zero coefficient isthe final one in the block, and of the number of consecutive zeros (RUN)and the level of the successive non-zero coefficients (LEVEL). Thedecoder side carries out the variable length decoding of the coded datato obtain the combination (LAST, RUN and LEVEL) so that it can reproducethe AC coefficients in the block. Incidentally, when carrying out thevariable length coding of the combination (LAST, RUN and LEVEL),although the MPEG-4 performs the variable length coding using differentVLC (Variable Length Coding) tables in accordance with the macroblocktype, the H.263 carries out the variable length coding using the sameVLC table independently of the macroblock type.

Thus, in the image decoding apparatus in the present embodiment, the ACcoefficient VLD table switching section 106 switches the AC coefficientVLD table in response to the macroblock type 313 fed from the macroblocktype/valid color difference block identification information decoder 87and the H.263 compatible identification information 33 fed from thecoding scheme decision section 32. When the H.263 compatibleidentification information 33 is set at the H.263, the AC coefficientdata variable length decoder 107 carries out the variable lengthdecoding using the single VLD table regardless of the macroblock type(intra or inter) 313, and supplies the decoded AC coefficient data 112to the texture decoder 10 as the coded texture data 6.

The coding scheme in the case where the combination (LAST, RUN andLEVEL) is not present in the VLC table also differs in the MPEG-4 and inthe H.263. When the combination (LAST, RUN and LEVEL) is not present inthe VLC table, the MPEG-4 encodes the Escape code followed by the valuecorrection of the RUN or LEVEL, and carries out either the variablelength coding or the fixed length coding. In contrast, the H.263 encodesthe Escape code, and then carried out the fixed length coding of thevalues of the LAST, RUN and LEVEL.

Thus, in the image decoding apparatus of the present embodiment, whenthe AC coefficient data variable length decoder 107 detects the Escapecode in the AC coefficient coded data, it supplies the coded bit streamto the switching section 108. When the H.263 compatible identificationinformation 33 is set at the H.263, the coded bit stream is supplied notto the AC coefficient data Esc coding decoder 110 but to the ACcoefficient data fixed length decoder 109 so that it carries out thefixed length decoding of the subsequent code about the LAST, RUN andLEVEL in their predetermined code length, and supplies the decoded ACcoefficient data 112 to the texture decoder 10 as the texture coded data6.

By the foregoing operation, the texture coded data 6 and the motionvector (texture motion data 7) output from the macroblock layer syntaxanalyzer 22 are delivered to the texture decoder 10 and the motioncompensator 8, respectively.

As described above, the syntax analysis/variable length decoder 2 asshown in FIG. 3 decodes and establishes the VOP prediction mode. Whenthe VOP prediction mode is inter, the differential vector in the texturemotion vector is decoded. The differential vector in the texture motionvector decoded is the differential vector between the prediction vectorobtained from motion vectors of three neighboring macroblocks and theactual motion vector. Thus, the motion vector (texture motion data 8) iscalculated by adding the differential vector of the motion vector to theprediction vector.

The prediction vector is calculated from the motion vectors of the threeneighboring macroblocks (MV1, MV2 and MV3), which have already beendecoded as shown in FIG. 14( a). When any one of the three neighboringmacroblocks is located outside the VOP, the motion vector of themacroblock outside the VOP is placed at the zero vector as shown in FIG.14( b) or 14(d). Alternatively, it can be set using the motion vector ofthe same macroblock in the VOP as shown in FIG. 14( c). However, whenthe coding scheme is H.263, and the GOB header is defined, it isnecessary for the prediction vector to be set within the boundary of theGOB. The prediction vector is set as in the VOP. In response to thedecoded vector, the prediction vector is extracted as the predictiontexture data 9 to be output to the texture decoder 10.

In contrast, when the VOP prediction mode is intra, the motioncompensation prediction is not carried out.

The texture decoder 10 receives the texture coded data 6, and restoresthe texture data 11.

FIG. 15 is a block diagram showing a configuration of the texturedecoder 10. An inverse quantizer 114 carries out the inversequantization of the texture coded data 6.

FIG. 16 is a block diagram showing a configuration of the inversequantizer 114.

A switching section 117 is switched in response to the macroblock type313 contained in the texture coded data 6. Because the texture codeddata 6 is not included in the DC coefficient data when the macroblocktype 313 of the block to be decoded is the inter coded mode, the texturecoded data 6 is directly supplied to an AC coefficient inverse quantizer120. In contrast, when the macroblock type 313 of the block to bedecoded is intra coded mode, the texture coded data 6 is supplied to theswitching section 118.

The switching section 118 is switched in response to the H.263compatible identification information 33. When the H.263 compatibleidentification information 33 indicates the MPEG-4 compatible H.263coded bit stream 203, a DC coefficient linear inverse quantizer 119Bcarries out the inverse quantization of the DC coefficient datacontained in the texture coded data 6. On the other hand, when the H.263compatible identification information 33 indicates the MPEG-4 coded bitstream 204, the DC coefficient non-linear inverse quantizer 119A carriesout the inverse quantization of the DC coefficient data, and outputs aDC coefficient 306. The DC coefficient quantization is carried out bydividing the DC coefficient by a predetermined value (calledquantization scale), and by dropping the fractional portion. Therefore,the decoding side can restore the DC coefficient 306 by multiplying thequantization DC coefficient by the quantization scale. The DCcoefficient linear inverse quantizer 119B differs from the DCcoefficient non-linear inverse quantizer 119A in the setting of thevalue of the quantization scale. The DC coefficient linear inversequantizer 119B carries out the inverse quantization using a fixed value8 as the quantization scale. In contrast, the DC coefficient non-linearinverse quantizer 119A non-linearly establishes the value of thequantization scale in accordance with the range of the quantization stepsize 319, and carries out the inverse quantization using thequantization scale, thereby outputting the DC coefficient 306.

The AC coefficient inverse quantizer 120 carries out the inversequantization of the AC coefficient data, and outputs an AC coefficient307. The DC coefficient 306 (which is present only in the intra codedmode) passing through the inverse quantization and the AC coefficient307 are transferred to an inverse DCT section 115 as a DCT coefficient308 which undergoes the inverse DCT, and is output as a decodedprediction error signal 309. An adder 116 adds the decoded predictionerror signal 309 to the prediction texture data 9 obtained by the motioncompensator 8, and outputs the sum as the decoded texture data 11. Theaddition of the prediction texture data 9 is not performed in the intracoded mode.

When the coded bit stream includes the H.263 compatible identificationinformation 33 multiplexed thereinto, it may sometimes include anend-of-sequence code (EOS) 227 indicating the end of the sequence, whichis multiplexed thereinto as shown in FIG. 2( a). The end-of-sequencecode 227 is detected by the picture start code detector 41 so that thedecoding operation is completed on the detection of the end-of-sequencecode 227.

As described above, the present embodiment 1 is configured such that itreceives the MPEG-4 compatible H.263 coded bit stream 203 consisting ofthe H.263 coded bit stream 201 into which the VO start code 231, VOLstart code, VO identification number 232 and H.263 compatibleidentification information 226 are multiplexed, and decodes theseinformation items. This offers an advantage of being able to implementan image decoding apparatus having compatibility between the H.263 andMPEG-4.

Embodiment 2

FIG. 17 is a block diagram showing a configuration of an image codingapparatus in the embodiment 2, which generates a coded bit streamdecodable by the image decoding apparatus described in the embodiment 1.In this figure, the reference numeral 121 designates an input imagesignal; 122 designates an H.263 encoder; 123 designates an H.263 codedbit stream; 124 designates an MPEG-4 compatible flag; 125 designates aheader information multiplexer; and 126 designates an MPEG-4 compatibleH.263 coded bit stream.

Next, the operation will be described.

First, the H.263 encoder 122 encodes the input image signal 121according to the H.263 syntax, and generates the H.263 coded bit stream123. Subsequently, the header information multiplexer 125, receiving theMPEG-4 compatible flag 124 indicative of generating the bit streamdecodable by an MPEG-4 based decoder, multiplexes, before the pictureheader of the H.263 bit stream, the VO start code 231, VO identificationnumber 232, VOL start code 233 and H.263 compatible identificationinformation (a flag of “0” or “1” indicative of the H.263 based bitstream) 226, which are needed for implementing decoding by the imagedecoding apparatus as described in the embodiment 1. Thus, the contentsof the MPEG-4 compatible H.263 coded bit stream 126 passing through themultiplexing become the bit stream as shown in FIG. 2( a) described inconnection with the embodiment 1.

When the H.263 coding apparatus 127 is carrying out real timecommunication with an MPEG-4 decoding apparatus 128 as shown in FIG. 18,the MPEG-4 decoding apparatus 128 can send the MPEG-4 compatible flag124 to the H.263 coding apparatus 127, and in response to the receptionof the MPEG-4 compatible flag 124, the H.263 coding apparatus 127 canmultiplex into the H.263 bit stream 123 the VO start code 231, VOidentification number 232, VOL start code 233 and H.263 compatibleidentification information 226, which are required for achievingdecoding by the image decoding apparatus as described in the embodiment1.

As described above, the present embodiment 2 multiplexes the VO startcode 231, VO identification number 232, VOL start code 233 and H.263compatible identification information 226 into the H.263 coded bitstream 123. This offers an advantage of being able to implement an imagecoding apparatus capable of generating a coded bit stream decodable byan MPEG-4 compatible image decoding apparatus.

Embodiment 3

FIG. 19 is a diagram showing a structure of an MPEG-4 compatible H.263coded bit stream 205 in the present embodiment 3. It includes inaddition to the conventional H.263 coded bit stream 201 as shown in FIG.1( a) a VO start code 231, a VO identification number 232 and an H.263start code 228. The H.263 start code 228 has the functions of both theVOL start code 233 and H.263 compatible identification information 226which are multiplexed in the embodiment 1.

The MPEG-4 coded bit stream 202 is identical to the conventional one asshown in FIG. 1( b).

The image decoding apparatus in the present embodiment differs from theimage decoding apparatus described in the embodiment 1 only in theheader information analyzer 21. FIG. 20 is a block diagram showing aconfiguration of the header information analyzer 21 in the embodiment 3.In this figure, the reference numeral 131 designates an H.263 startcode/VOL start code detector; and 132 designates a coding schemedecision section. The VO start code detector 30, H.263 compatibleidentification information 33, switching section 34, H.263 pictureheader information analyzer 35, H.263 GOB header information analyzer36, VOL header information decoder 37 and VOP header informationanalyzer 38 are the same as their counterparts as shown in FIG. 5 of theembodiment 1.

Next, the operation will be described.

In response to the detection of the VO start code 231 by the VO startcode detector 30, the following decoding operation is started. First, asfor the MPEG-4 compatible H.263 coded bit stream 205, the H.263 startcode/VOL start code detector 131 detects the H.263 start code, while asfor the MPEG-4 coded bit stream 202, it detects the VOL start code 233.

In the MPEG-4, the start code in each layer consists of a code (00000000 0000 0000 0000 0001) common to all the start codes, followed by afixed length (5-bit) start code unique to the layer. The common startcode is surely detected as the start code in the bit stream. Thus, theH.263 start code 228 also has a structure consisting of the common startcode followed by the fixed length (5-bit) code enabling it to beidentified as the H.263 coded bit stream.

When the start code detected is the H.263 start code 228, the codingscheme decision section 132 places the H.263 compatible identificationinformation 33 at the H.263. In contrast, when the start code is the VOLstart code 233, it places the H.263 compatible identificationinformation 33 at the MPEG-4. The subsequent operation is the same asthat of the embodiment 1.

As described above, the present embodiment 3 is configured such that itreceives the MPEG-4 compatible H.263 coded bit stream 205 consisting ofthe H.263 coded bit stream 201 into which the VO start code 231, VOidentification number 232 and H.263 start code 228 are multiplexed, anddecodes these information items. This offers an advantage of being ableto implement an image decoding apparatus having compatibility betweenthe H.263 and MPEG-4.

Embodiment 4

The present embodiment 4 is an image coding apparatus for generating abit stream decodable by the image decoding apparatus described in theembodiment 3, and has the same configuration as that shown in FIG. 17 ofthe embodiment 2.

Next, the operation will be described.

First, the H.263 encoder 122 encodes the input image signal 121according to the H.263 syntax, thereby generating the H.263 coded bitstream 123. Subsequently, receiving the MPEG-4 compatible flag 124, theheader information multiplexer 125 multiplexes, before the pictureheader of the H.263 bit stream, the VO start code 231, VO identificationnumber 232 and H.263 start code 228, which are needed for implementingdecoding by the image decoding apparatus as described in the embodiment3. Thus, the contents of the MPEG-4 compatible H.263 coded bit stream126 passing through the multiplexing become equivalent to those of thebit stream as shown in FIG. 19 described in connection with theembodiment 3.

Incidentally, the MPEG-4 compatible flag 124 can be transferred from theMPEG-4 decoding apparatus 128 as described in connection with FIG. 18 ofthe embodiment 2.

As described above, the present embodiment 4 multiplexes the VO startcode 231, VO identification number 232 and H.263 start code 228 into theH.263 bit stream 201. This offers an advantage of being able toimplement an image coding apparatus capable of generating a coded bitstream decodable by the MPEG-4 compatible image decoding apparatus.

Embodiment 5

The present embodiment 5 comprises a multiplexer for multiplexing theheader information for implementing the MPEG-4 compatibility on anetwork, for example, independently of a coding apparatus. FIG. 21 is adiagram showing an image communications system in the present embodiment5. In this figure, the reference numeral 141 designates an H.263 codingapparatus; 142 designates an MPEG-4 decoding apparatus; and 143designates a coded bit stream converting apparatus. The H.263 codingapparatus 141, MPEG-4 decoding apparatus 142 and coded bit streamconverting apparatus 143 are connected to a network.

Next, the operation will be described.

Receiving an MPEG-4 compatible flag 147 requesting for an MPEG-4compatible H.263 coded bit stream 148 from the MPEG-4 decoding apparatus142 or from a user, the coded bit stream converting apparatus 143receives an H.263 coded bit stream 146 from the H.263 coding apparatus141, multiplexes into the H.263 coded bit stream 146 the headerinformation needed by the MPEG-4 decoding apparatus for carrying outdecoding as described in the embodiment 2 or 4, and transmits themultiplexed data to the MPEG-4 decoding apparatus 142.

As described above, the present embodiment 5 comprises the coded bitstream converting apparatus 143 in the network. This offers an advantageof being able to implement an image communications system havingcompatibility between the H.263 and MPEG-4.

Embodiment 6

FIG. 22 is a diagram showing an image communications system in thepresent embodiment 6. In this figure, the reference numeral 141designates the H.263 coding apparatus; 143 designates the coded bitstream converting apparatus; 144 designates a server; and 145 designatesan MPEG-4 decoding apparatus built-in type browser, which are connectedto a network.

Next, the operation will be described.

When the MPEG-4 decoding apparatus built-in type browser 145 makes anaccess to the H.263 coded bit stream 146 transmitted on the network, ittransmits to the server 144 the MPEG-4 compatible flag 147 indicative ofdecoding by the MPEG-4 decoding apparatus. Receiving the MPEG-4compatible flag 147, the server 144 transmits the H.263 coded bit stream146 to the coded bit stream converting apparatus 143.

The coded bit stream converting apparatus 143 generates the MPEG-4compatible H.263 coded bit stream 148 decodable by the MPEG-4 decodingapparatus by adding header information to the received H.263 bit stream146 as described in the embodiment 2 or 4, and transmits it to theMPEG-4 decoding apparatus built-in type browser 145. Receiving theMPEG-4 compatible H.263 coded bit stream 148, the MPEG-4 decodingapparatus built-in type browser 145 can decode the H.263 coded bitstream 146 to display images.

The MPEG-4 decoding apparatus built-in type browser 145 itself can alsoincorporates the coded bit stream converting apparatus 143. In thiscase, the MPEG-4 decoding apparatus built-in type browser 145 receivesthe H.263 coded bit stream 146 from the server 144, and converts theMPEG-4 compatible H.263 coded bit stream 148, so that the built-inMPEG-4 decoding apparatus can decode it to display images.

As described above, the present embodiment 6 comprises on the networkthe coded bit stream converting apparatus and the server. This offers anadvantage of being able to implement an image communications systemhaving compatibility between the H.263 and MPEG-4.

Embodiment 7

The image decoding apparatuses as described in the foregoing embodiments1 and 3 can distinguish the H.263 bit stream from the MPEG-4 bit stream.However, they cannot receive the H.263 bit stream as it is because theheader information for making it MPEG-4 compatible must be multiplexedinto the initial position of the H.263 bit stream generated by the H.263coding apparatus. The embodiment 7 is an image decoding apparatuscapable of receiving the H.263 bit stream without any change.

FIG. 23 is a block diagram showing a configuration of the headerinformation analyzer 21 in the present embodiment 7. In this figure, thereference numeral 151 designates an H.263 picture start code detectorfor detecting an H.263 picture start code 221 multiplexed into the H.263coded bit stream; 152 designates a coding scheme decision section; and153 designates an H.263 picture header information analyzer for settingthe VOL header information and VOP header information in response to thepicture header information 222 multiplexed into the H.263 coded bitstream. The remaining VO start code detector 30, H.263 compatibleidentification information 33, switching section 34, H.263 GOB headerinformation analyzer 36, VOL header information decoder 37 and VOPheader information analyzer 38 correspond to those of the embodiment 1.The components other than the header information analyzer 21 areequivalent to those of the image decoding apparatus of the embodiment 1.

Next, the operation will be described.

The H.263 picture start code detector 151 always monitors the start andend of the coded bit stream as shown in FIGS. 24( a) and 24(b). Itmonitors, as a continuous coded bit stream, from the picture start code221 to the macroblock data 225 as for the H.263 coded bit stream 201,while from the VO start code 231 to the macroblock data 239 with theMPEG-4 coded bit stream 202.

Receiving the H.263 coded bit stream 201, the H.263 picture start codedetector 151 detects the picture start code 221, and supplies the resultto the coding scheme decision section 152. The coding scheme decisionsection 152 makes a decision from the picture start code 221 that thereceived coded bit stream is the H.263 coded bit stream 201, and placesthe H.263 compatible identification information 33 at the H.263. Incontrast, when the VO start code detector 30 detects the VO start code231, the coding scheme decision section 152 makes a decision that thereceived coded bit stream is the MPEG-4 coded bit stream 202, and placesthe H.263 compatible identification information 33 at the MPEG-4.

As for the H.263 coded bit stream 201, the switching section 34 suppliesit to the H.263 picture header information analyzer 153. The H.263picture header information analyzer 153 decodes the picture headerinformation 222 multiplexed into the H.263 coded bit stream 201, andsets the VOL header information and VOP header information as in theembodiment 1. The subsequent operation is the same as that of theembodiment 1.

On the other hand, as for the MPEG-4 coded bit stream 202, the switchingsection 34 supplies it to the VOL header information decoder 37. Thesubsequent operation is the same as that of the embodiment 1.

As described above, the present embodiment 7 decides that the bit streamis the H.263 coded bit stream 201 when detecting the picture start code221, and sets the VOL header information and VOP header information.This offers an advantage of being able to implement an image decodingapparatus having compatibility between the H.263 and MPEG-4.

Embodiment 8

The present embodiment 8 relates to a coded bit stream convertingapparatus for converting the H.263 coded bit stream 201 as shown in FIG.1( a) to the MPEG-4 coded bit stream 202 as shown in FIG. 1( b).

FIG. 25 is a block diagram showing the coded bit stream convertingapparatus in the embodiment 8. In this figure, the reference numeral 161designates a syntax analyzer for splitting the H.263 coded bit stream201 into a picture header information code word 401, GOB headerinformation code word 402 and macroblock data code word 403; 162designates a picture header information decoder for decoding the pictureheader information code word 401; 163 designates a GOB headerinformation analyzer/converter for decoding the GOB header informationcode word 402; 164 designates an MPEG-4 header information settingsection for setting the VOL header information 234 and VOP headerinformation 236; and 165 designates a multiplexer for producing theMPEG-4 coded bit stream 202.

Next, the operation will be described.

The syntax analyzer 161, detecting the picture start code 221 in theH.263 coded bit stream 201, splits the subsequent coded bit stream intothe picture header information code word 401, GOB header informationcode word 402 and macroblock data code word 403, and supplies them tothe picture header information decoder 162, GOB header informationanalyzer/converter 163 and multiplexer 165. The GOB header informationcode word 402 is not necessarily multiplexed into the H.263 coded bitstream 201, but is multiplexed as long as the GOB start code 223 isdetected. When the GOB start code 223 is detected, GOB header detectioninformation 404 is supplied to the MPEG-4 header information settingsection 164. The picture header information decoder 162 decodes thepicture header information code word 401 as in the embodiment 1, andsupplies the MPEG-4 header information setting section 164 with pictureheader information 405 decoded.

In response to the decoded picture header information 405, the MPEG-4header information setting section 164 sets the VOL header information234 and VOP header information 236 as in the embodiment 1. As with theheader information not referred to in the embodiment 1, any valuedisclosed in the ISO/IEC JTC1/SC29/WG11 MPEG-4 Video VM8.0 can be set.When the MPEG-4 header information setting section 164 receives the GOBheader detection information 405, it enables the error resistant codinginstruction mode.

As described in the embodiment 1, the decoding procedure of themacroblock data of the H.263 differs from that of the MPEG-4.Accordingly, the decoding side must change the decoding method inresponse to switching information. For this reason, the followingswitching information must be set in the VOL header.

(1) AC Coefficient VLC Table Switching Information.

Information for switching VLD tables used for carrying out the variablelength decoding of the AC coefficient data on the decoding side, whenthe coding side uses different VLC tables for carrying out the variablelength coding of the AC coefficient data as described in the embodiment1.

(2) Esc Coding Switching Information.

Information for switching decoding schemes on the decoding side, whenthe coding side uses different coding schemes in the case where the ACcoefficient data is not present in the VLC tables when carrying out thevariable length coding of the AC coefficient data as described in theembodiment 1.

(3) Intra DC Coefficient Inverse Quantization Switching Information.

Information for switching the inverse quantization method of the DCcoefficients, when the coding side employs different intra DCcoefficient quantization methods as described in the embodiment 1.

The switching information items of the foregoing (1)-(3) can beintegrally set as information for switching between the techniqueemployed by the H.263 and other techniques.

The MPEG-4 header information set by the MPEG-4 header informationsetting section 164 undergoes the variable length coding, and issupplied to the multiplexer 165 as MPEG-4 header information code word406.

The GOB header information analyzer/converter 163 decodes the GOB headerinformation code word 402 as in the embodiment 1, and converts the GOBheader information 224 into the resynchronization information 238 in theMPEG-4 representation form.

The MPEG-4 resynchronization information 238 is used as an errorresistance reinforcer, and is multiplexed when the error resistantcoding indication information of the VOL header information 236 isvalid. When decoding the resynchronization information 238, the decodingside establishes the resynchronization with the coded bit stream, andresets the prediction vector and quantization step size used fordecoding the macroblock. In the H.263, the prediction vector and thequantization step size are reestablished when the GOB header information224 is decoded. Therefore, converting the GOB header information 224using the resynchronization information 238 enables the GOB headerinformation 224 to be converted into the MPEG-4 representation form.

FIG. 26 is a diagram showing a structure of the GOB header information224 and the resynchronization information 238. A macroblock number 271in the resynchronization information 238 is the number indicating theposition of the macroblock in the VOP. It can be obtained by calculatingthe position of the macroblock corresponding to the received H.263macroblock data in the picture. Since it corresponds to the firstmacroblock in the GOB, it can be calculated from the GOB number. Aquantization scale 272 is obtained by setting the GOB quantization stepsize. A header expansion instruction code 273 is “1” when a timereference 274 and a VOP elapsed time 275 are to be multiplexed. Theseitems of the information are used for representing the individual VOPs.The time reference 274 and the VOP elapsed time 275 can be set as neededwhen setting the header expansion instruction code 273 at “1”. Theresynchronization information 238 undergoes the variable length coding,so that the multiplexer 165 is supplied with a resynchronizationinformation code word 407 that includes a resynchronization instructioncode, that is, a fixed length unique code indicating that theresynchronization information 238 is multiplexed.

The multiplexer 165 multiplexes the MPEG-4 header information code word406, resynchronization information code word 407 and macroblock datacode word 403 into the coded bit stream, and supplies it to the MPEG-4coded bit stream 202.

Although the resynchronization information is assumed to be multiplexedwhen the error resistant coding indication information of the VOL headerinformation 234 is valid in the present embodiment, it can bemultiplexed regardless of whether the error resistant coding indicationinformation is valid or invalid.

The syntax analyzer 161 completes its analysis when it detects theend-of-sequence code 227 in the case where the end-of-sequence code 227is added after the macroblock data 225 in the H.263 coded bit stream201.

As described above, the present embodiment converts the H.263 coded bitstream 201 into the MPEG-4 coded bit stream 202. This offers anadvantage of being able to decode the H.263 coded bit stream by theMPEG-4 image decoding apparatus.

Embodiment 9

Although in FIG. 23 of the embodiment 7 when the H.263 picture startcode detector 151 detects the picture start code 221, the coding schemedecision section 152 identifies the H.263 coded bit stream 201, and theH.263 picture header information analyzer 153 sets the VOL headerinformation and VOP header information, the present embodiment switchesthe operation of the macroblock layer syntax analyzer 22 in response tothe picture header information 222 decoded by the H.263 picture headerinformation decoder 42 as shown in FIG. 6, which is included in theH.263 picture header information analyzer 153. This can obviate theMPEG-4 header information setting section 43. In addition, when the GOBstart code detector 61 as shown in FIG. 8 of the embodiment 7 detectsthe GOB start code 223 in the H.263 coded bit stream 201, the GOB headerinformation decoder 62 decodes the GOB header information 224, and theMPEG-4 header information update section 63 resets the VOP quantizationstep size included in the VOP header information 236. However, to decodethe H.263 coded bit stream 201, it is enough for the present embodimentto reset the picture quantization step size 304 included in the pictureheader information 222 in order to decode the macroblock data using thepicture header information 222.

Next, the operation of the macroblock layer syntax analyzer 22 will bedescribed when decoding the macroblock data in response to the pictureheader information 222 decoded by the H.263 picture header informationdecoder 42.

Since the present embodiment differs in the operation of the switchingsections 81, 83, 88 and 95, in the operation of the adder 94 and in theoperation of the motion vector decoder 97 in the macroblock layer syntaxanalyzer as shown in FIG. 12, and differs in the operation of theswitching section 102 in the block data decoder 98 as shown in FIG. 13,only the different portions will be described.

When the MPEG-4 coded bit stream 202 is decoded, that is, when theMPEG-4 is designated by the H.263 compatible identification information33 that is set by the coding scheme decision section 152 as shown inFIG. 23, the switching section 81 is switched in response to thegeometry information decoded by the VOL header information decoder 37.In contrast with this, when the H.263 coded bit stream 201 is decoded,that is, when the H.263 compatible identification information 33indicates the H.263, the bit stream 1 is unconditionally supplied to theswitching section 83 without passing through the geometry coded datadecoder 82.

When the MPEG-4 coded bit stream 202 is decoded, the switching section83 is switched in response to the VOP prediction type decoded by the VOPheader information analyzer 38. On the other hand, when the H.263 codedbit stream 201 is decoded, the switching section 83 is switched inresponse to the picture coding type 302 decoded by the H.263 pictureheader information decoder 42. The switching operation itself is thesame as that of the embodiment 1, and is carried out in response towhether the picture coding type 302 is intra or not.

When the MPEG-4 coded bit stream 202 is decoded, the switching section88 is switched in response to the intra AC/DC prediction indicationinformation decoded by the VOL header information decoder 37. When theH.263 coded bit stream 201 is decoded, that is, when the H.263compatible identification information 33 indicates the H.263, the bitstream 1 is unconditionally supplied to the valid block identificationinformation decoder 90 without passing through the AC predictionindication information decoder 89.

When the MPEG-4 coded bit stream 202 is decoded, the adder 94 adds tothe decoded differential quantization step size 254 the VOP quantizationstep size of the first previous macroblock decoded, and outputs the sumas the quantization step size. In contrast with this, when the H.263coded bit stream 201 is decoded, it adds to the decoded differentialquantization step size 254, the picture quantization step size of thefirst previous macroblock decoded, and outputs the sum as thequantization step size.

When the MPEG-4 coded bit stream 202 is decoded, the switching section95 is switched in response to the interlace mode indication informationdecoded by the VOP header information analyzer 38. When the H.263 codedbit stream 201 is decoded, that is, when the H.263 compatibleidentification information 33 indicates the H.263, the bit stream 1 isunconditionally supplied to the motion vector decoder 97 without passingthrough the interlace information decoder 96.

When the MPEG-4 coded bit stream 202 is decoded, the motion vectordecoder 97 decodes the motion vector (texture motion data 7) in responseto the motion vector search range designation information decoded by theVOP header information analyzer 38. When the H.263 bit stream isdecoded, the motion vector decoder 97 decodes the motion vector (texturemotion data 7) in response to the motion vector search range defined bythe H.263.

When the MPEG-4 coded bit stream 202 is decoded, the switching section102 in the block data decoder 98 is switched in response to the intraAC/DC prediction mode indication information decoded by the VOL headerinformation decoder 37. When the H.263 coded bit stream 201 is decoded,that is, when the H.263 compatible identification information 33indicates the H.263, the bit stream 1 is unconditionally supplied to theDC coefficient fixed length decoder 103. The subsequent operation is thesame as that of the embodiment 1.

As described above, the embodiment 9 is configured such that it makes adecision that the bit stream is the H.263 coded bit stream 201 when itdetects the picture start code 221, decodes the picture headerinformation 222, and decodes the macroblock data in response to thepicture header information 222 decoded. This offers an advantage ofbeing able to implement the image coding apparatus having compatibilitybetween the H.263 and MPEG-4 without setting the VOL header informationand VOP header information.

INDUSTRIAL APPLICABILITY

As described above, the image decoding apparatus, image codingapparatus, image communications system and coded bit stream convertingapparatus in accordance with the present invention can transmit andreceive the coded bit stream of a different coding scheme in a simpleconfiguration.

1. An image decoding apparatus for decoding at least a first coded bitstream into which header information of the H.263 coding scheme andimage coded data encoded in the H.263 coding scheme are multiplexed, orfor decoding a second coded bit stream into which header information ofthe MPEG-4 coding scheme and image coded data encoded in the MPEG-4coding scheme are multiplexed, the image decoding apparatus comprising:a coding scheme decision means for making a decision as to whether areceived coded bit stream is the first coded bit stream or the secondcoded bit stream in response to the first header information or to thesecond header information; a setter for setting VOL geometry informationin image coding information to be used for decoding the received firstcoded bit stream according to a syntax of the second coded bit stream toa rectangle shape; and a decoder for decoding image coding informationincluded in the header information of the MPEG-4 coding scheme from thereceived second coded bit stream, wherein when the first coded bitstream is received, the received first coded bit stream is decodedaccording to the syntax of the second coded bit stream based on theimage coding information set by the setter, and when the second codedbit stream is received, the received second coded bit stream is decodedaccording to the image coding information decoded by the decoder.